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ABSTRACT 

We focus on the problem of differentially private histogram 
publication, for range-sum query answering. Specifically, we 
derive a histogram from a given dataset, such that (i) it sat¬ 
isfies e-differential privacy, and (ii) it achieves high utility for 
queries that request the sum of contiguous histogram bins. 
Existing schemes are distinguished into two categories: fast 
but oblivious to utility optimizations that exploit the data 
characteristics, and data-aware but slow. We are the first to 
address this problem with emphasis on both efficiency and 
utility. Towards this goal, we formulate a principled ap¬ 
proach, which defines a small set of simple modules, based 
on which we can devise a variety of more complex schemes. 
We first express the state-of-the-art methods in terms of 
these modules, which allows us to identify the performance 
bottlenecks. Next, we design novel efficient and effective 
schemes based on non-trivial module combinations. We ex¬ 
perimentally evaluate all mechanisms on three real datasets 
with diverse characteristics, and demonstrate the benefits of 
our proposals over previous work. 


1. INTRODUCTION 

A histogram computed on a dataset D is a vector of counts, 
such that each record in D affects at most a single histogram 
element, called bin. The histogram constitutes one of the 
most basic statistical tools for describing the dataset distri¬ 
bution. A range-sum query over a histogram returns the 
sum of values of a set of contiguous bins. Our goal is to 
publish a histogram on D that satisfies e-differential privacy 
jdj. This paradigm entails perturbing the bins prior to their 
publication, so that each individual record in D is protected. 
We aim at minimizing the total error incurred by the per¬ 
turbation when answering arbitrary (i.e., not known a pri¬ 
ori) range-sum queries. For example, consider a database D 
with medical records, and a histogram on D where each bin 
contains the number of patients of a certain age. Assuming 
bins sorted on age, a range-sum query over this histogram 


returns the number of patients in the range. In this sce¬ 
nario, it is important that the published histogram does not 
violate the privacy of any individual patient; at the same 
time the range-sum results should be accurate. 

Differentially private histogram publication has been stud¬ 
ied extensively. The existing schemes can be divided into 
two categories. “Data-aware” methods exploit the under¬ 
lying dataset distribution [ 5 ] |20| |22| |13| . They smooth the 
histogram prior to its perturbation by grouping similar bins 
and replacing their values with the group average. This 
yields reduced perturbation per bin. However, these mech¬ 
anisms exhibit superquadratic time complexities, which may 
be prohibitive in time-critical applications. “Data-oblivious” 
methods build a perturbed aggregate tree on top of the his¬ 
togram, and answer range-sum queries by summing a small 
number of tree nodes, instead of numerous individual bins 
falling in the range [TJ [lO, 17][l9]. Such approaches are very 
efficient, running in time linear to the number of histogram 
bins, but may yield low utility for some practical datasets. 

To the best of our knowledge, this is the first work that 
aims at efficiency, without compromising utility. Towards 
this goal, we address the problem in a principled, modu¬ 
lar approach. Specifically, we first identify three building 
blocks, which we call modules, namely Smoothing, Hierar¬ 
chy Level, and Fixed Queries', the first is inspired by the 
data-aware techniques, the second works on a single level 
in the aggregate tree, and the third is based on techniques 
such as the matrix mechanism 14], which has been applied 
to increase utility for fixed query workloads 13 . We then 


formulate a scheme as a combination of these modules, in¬ 
tegrated with certain components, called connectors. The 
latter do not affect privacy, but serve to properly format 
the inputs of the modules. Subsequently, we express the ex¬ 
isting state-of-the-art methods in our modular framework, 
and discover opportunities for optimization. Finally, we de¬ 
vise novel efficient and effective schemes by composing the 
modules non-trivially. Concretely, our contributions are: 


• We introduce a modular framework on differentially 
private histograms for range-sum queries. Our ap¬ 
proach offers multiple benefits: (i) using a small set of 
simple modules and connectors, we can analyze all ex¬ 
isting methods, and devise novel schemes with variable 
efficiency and utility, (ii) given the privacy level of each 
module, we can easily derive the privacy of arbitrarily 
complex schemes, and (iii) each module can be opti¬ 
mized separately; furthermore, potential future opti¬ 
mizations can be incorporated to an existing scheme 
with minimal effort. 
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• We analyze an important submodule of the Smoothing 
module, namely the grouping method, which essen¬ 
tially solves an optimization problem. We point out 
the two objective functions most heavily used in the 
literature, propose optimizations and evaluate their ef¬ 
fect on utility. 

• We design novel schemes based on the defined modules, 
including the first mechanism that seamlessly com¬ 
bines the data-aware and -oblivious methodologies. In 
addition, we efficiently adapt schemes for fixed query 
workloads (e.g., the matrix mechanism) to arbitrary 
range-sums, via a simple but powerful technique based 
on prefix sums |ll] . 


Definition 1. A mechanism M : H —> 1-1 satisfies e- 
differential privacy for a histogram algorithm F £ F , if 
for all sets H C it, and every pair D,D' £ V where D' 
is obtained from D by removing a record (D,D' are called 
neighboring), it holds that 

Pr [M(F(D)) £ H) < e £ ■ Pr[M{F(D')) £ H] 

Intuitively, e-differential privacy guarantees that the per¬ 
turbed histogram H will be the same with high probability 
(tunable by e), regardless of whether a patient agrees to 
participate in the publication or not. Equivalently, the sen¬ 
sitive information of any patient cannot be inferred from the 
published data. 


• We provide a thorough experimental evaluation that 
compares the best existing methods with our new so¬ 
lutions, testing over three real datasets with different 
characteristics. We exhibit that there is a trade-off 
between algorithmic efficiency and utility across the 
various approaches. 

The remainder of the paper is organized as follows. Sec¬ 
tion [2] includes preliminary information and surveys the re¬ 
lated work. Section [3] introduces our modular framework. 
Section [f] investigates grouping in depth. Section [5] presents 
our proposed mechanisms. Section [5] experimentally evalu¬ 
ates all schemes, whereas Section [ 7 ] concludes our work. 


2. BACKGROUND 

Section [2.1| formulates our setting, and includes the nec¬ 
essary primitives on differential privacy. Section [2.2| surveys 
differentially private histogram publication. 

2.1 Setting and Primitives 

Let V be a collection of datasets. We define a family 
of functions T = {Fj : V —> TL}, such that for all j and 
all D £ T >, Fj(D) jt h £ TL is an (ordered) vector called 
histogram. An element of h is termed bin and consists of a 
value and a label, where h[i] represents the bin value of 

h. All histograms have the property that any record in D 
increments at most a single h[i] by 1. Finally, we call Fj a 
histogram algorithm. For instance, let D £ V be a dataset 
of medical records. Then, F\ £ T may produce histogram 
hi such that hi[i] is the number of patients in D having age 

i, and F 2 £ T may produce histogram I12 such that h2[i] is 
the number of patients in hospital with id i. Observe that, 
the presence of a patient in D increments at most one bin 
by 1 in both histograms. 

Our goal is to publish a n-element histogram h produced 
by some fixed algorithm F on a D £ V, while satisfying e- 
differential privacy and allowing arbitrary range-sum queries 
on its bins with high utility. Specifically, we define a range- 
sum query as a range of bins [ii,i u \, 1 < ii < iu < n, which 
returns the sum fTf'L, h[i]. In our example above, a range- 
sum query on hi could be [10, 20], asking for the number of 
patients between 10 and 20 years old. We assume that the 
range queries are not known prior to the publication of the 
histogram. 

To achieve e-differential privacy, we apply a mechanism M 
on the histogram, which perturbs it in a way that satisfies 
the following definition, adapted from [ 5 ]. 


Definition 2. The sensitivity of any histogram algo¬ 
rithm F £ T is A (F) = max C| Di £I) \\F(D) — F(D') || = 1 
for all neighboring D, D' £ V. 

In other words, the sensitivity of F represents how much 
the histogram F(D) changes when a record is deleted from 
D. Since any record contributes 1 to at most a single bin, 
the sensitivity is 1 for any histogram algorithm F £ T. 

The most basic technique to achieve e-differential privacy 
is to add Laplace noise to the histogram bins using the 
Laplace Perturbation Algorithm (LPA 0 [6])- Let Lap (A) 
be a random variable drawn from a Laplace distribution 
with mean zero and scale parameter A. LPA achieves e- 
differential privacy through the mechanism outlined in the 
following theorem, adapted from [7|. 

Theorem 1. Let F £ T and define h = f F(D). A mech¬ 
anism M that adds independently generated noise from a 
zero-mean Laplace distribution with scale parameter A = 
A (F)/e = 1/e to each of the values of h, i.e., which pro¬ 
duces transcript h = h + (Lap(l/e)) n , enjoys e-differential 
privacy. 


With LPA, a range-sum query [ii,i u \ is processed on the 
noisy h and returns Yli=i h[*]• The Laplace noise injected 
in each bin introduces error, which is aggregated when the 
noisy bin values are added. For large ranges, this error 
may completely destroy the utility of the answer. Numer¬ 
ous works (overviewed in Section 2.21 introduce alterna¬ 
tive mechanisms for improving the utility of the output his¬ 
tograms in the case of range-sum queries. 

Finally, we include a composition theorem (adapted from 
[16] ) that is useful for our proofs. It concerns executions of 
multiple differentially private mechanisms on non-disjoint 
and disjoint inputs. 


Theorem 2. Let Mi, ..., M r be mechanisms, such that 
each Mi provides a-differential privacy. Let hi, ..., h r £ TL 
be histograms created on pairwise non-disjoint (resp. dis¬ 
joint) datasets D\,..., D r , respectively. Let M be another 
mechanism that executes Mi (hi),..., M r (h r ) using inde¬ 
pendent randomness for each Mi, and returns their out¬ 
puts. Then, M satisfies (X][=i e D -differential privacy (resp. 
(max[_i ei)-differential privacy). 


The above theorem allows us to view e as a privacy budget 
that is distributed among the r mechanisms. Moreover, note 
that the theorem holds even when M; receives as input the 
private outputs of Mi,..., M;_i |16|. 
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2.2 Differentially Private Histograms 

Existing literature on differentially private histograms for 
range-sum queries aims at improving upon LPA in terms of 
utility. We divide the approaches into two categories; data- 
aware that utilize smoothing, and data-oblivious that rely 
on hierarchical tree structures. 

Data-aware methods. These approaches first smooth the 
histogram, typically either by grouping similar bin values 
and substituting them with their average, or by performing 
a smoothing filter such as the Discrete Fourier Transform 
(DFT). Subsequently, they apply Laplace noise similar to 
LPA to the averages or the DFT coefficients. Range-sum 
queries are processed by summing the histogram bin val¬ 
ues in the query range. Smoothing reduces the sensitivity 
and, hence, the injected Laplace noise, but adds approxima¬ 
tion error. Consequently, smoothing methods are effective 
if the Laplace noise error reduction exceeds the smoothing 
approximation error. The bin grouping algorithm assigns 
scores to a set of potential grouping strategies, and selects 
the one with the minimum score, in a manner that does not 
compromise differential privacy. Existing approaches differ 
in the set of examined strategies, the scoring function, and 
the selection process. 

The SF algorithm [20 follows the grouping and averag¬ 
ing paradigm. Specifically, given as input a fixed parameter 
k and privacy budgets e, t , SF initially finds a set of k 
groups of contiguous bins through an e'-differentially pri¬ 
vate process. Subsequently, it smooths the bin values based 
on the grouping, and adds Laplace noise generating (e — e)- 
differentially private histogram. Due to linear composition 
(Theorem [5|, the SF mechanism achieves e-differential pri¬ 
vacy. The grouping sub mechanism of SF operates on the 
original histogram and determines the k groups such that 
the estimated squared error is minimized. This error is ex¬ 
pressed as the sum of (i) the squared approximation error 
due to smoothing, and (ii) the squared error from injecting 
Laplace noise with scale l/(e — e) prior to publication. It 
then applies the exponential mechanism |15] in order to alter 
the group borders and achieve e'-differential privacy. Note 
that, due to this step, the total error of SF eventually devi¬ 
ates from the actual minimum. The grouping submodule of 
SF runs in O(n^). 

Acs et al. [[2 present two mechanisms, EFPA and P- 


expressed as a function of the noisy data, rather than the 
original histogram (and, thus, similar to 20 2], it does not 


HP. EFPA is an improvement of 18 , which smooths the 


histogram using a subset of its DFT coefficients perturbed 
with Laplace noise, while guaranteeing that the output his¬ 
togram satisfies e-differential privacy. P-HP is a grouping 
and averaging method that improves SF 20 . In particular, 


instead of receiving the number of groups k as input, it dis¬ 
covers the optimal value of k on-the-fly. Contrary to SF, it 
utilizes an absolute error metric. The grouping algorithm 
of P-HP runs also in 0(n 2 ), but similarly to SF does not 
examine all possible groups. P-HP is shown to outperform 
both EFPA and SF in terms of utility [2j. 

Motivated by 12j (for a different setting), AHP [22] first 
applies LPA to the histogram with scale 1/e, and sorts 
the resulting bins in descending or ascending order. Sub¬ 
sequently, it executes a grouping and averaging technique 
that is different from SF and P-HP. Specifically, it oper¬ 
ates on already e'-differentially private data and, hence, does 
not need to apply the exponential mechanism. Moreover, it 
finds the grouping that minimizes the squared error metric 


guarantee the actual minimum error). Note that the order¬ 
ing attempts to minimize the approximation error, since it 
results in groups with more uniform bin values. The au¬ 
thors present two algorithms; one that evaluates all possi¬ 
ble groups and runs in 0(n 3 ) time, and a greedy one that 
considers only a subset of the possible options and runs in 
0(n 2 ). They conducted experiments using the latter, and 
demonstrated that AHP offers better utility than P-HP. 

DAWA |13] comprises of two stages. The first stage exe¬ 
cutes a smoothing technique, while the second an optimized 
version of the matrix mechanism [14]. Its grouping and av¬ 
eraging submodule invests e budget to reduce the absolute 
error metric similar to [ 2 ]. However, instead of executing 
the exponential mechanism, it adds noise to the costs of the 
groups used in the selection process on-the-fly. The authors 
present two instantiations; the first evaluates all possible 
groupings and runs in 0(n 2 log n) time, whereas the second 
considers only a subset and runs in 0(nlog 2 n). The out¬ 
put of the smoothing procedure is fed to the matrix mech- 
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anism. The latter belongs to a category of schemes [14 
[ 9 ] that take as input a set of pre-defined range-sum queries 
and assign more privacy budget to the bins affecting nu¬ 
merous queries. DAWA can be adapted to our setting of 
arbitrary queries in two ways; either by completely ignoring 
the second stage, resulting in time complexity 0(n 2 log n) 
(or 0(n log 2 n) in the approximate version), or by feeding 
all the 0(n 2 ) possible queries to the input of the matrix 
mechanism, yielding time complexity 0(n 3 logn). 

Data-oblivious methods. These schemes build an aggre¬ 
gate tree on the original histogram; each bin value is a leaf, 
and each internal node represents the sum of the leaves in its 
subtree. In order to achieve e-differential privacy, they add 
Laplace noise to each node, which is proportional to the tree 
height (since each bin value is incorporated in all the sums 
along its path to the root). A range-sum query is processed 
by identifying the maximal subtrees that exactly cover the 
range, and summing the values stored in their roots. Com¬ 
pared to LPA, the hierarchical methods essentially increase 
the sensitivity from 1 to logn, but sum fewer noisy values 
when processing the range-sum, reducing the aggregate er¬ 
ror. For a range-sum covering m bins, these methods induce 
O(logmlogn) error, as opposed to LPA that inflicts 0(m) 
error. Therefore, the hierarchical methods exhibit benefits 
for large ranges. Moreover, their time complexity is O(n). 

Hay et al. [To] build a binary aggregate tree and inject 
Laplace noise uniformly across all nodes. In addition to con¬ 
structing the final range-sum from the roots of the maximal 
subtrees that cover the range, they also explore other node 
combinations. Independently from 10 , Privelet 19 builds 


a Haar wavelet tree and adds Laplace noise, achieving prac¬ 
tically the same effect as [0;. Based on the observation that 
the privacy budget should not be divided equally among all 
levels, Cormode et al. 4 enhance 10 with a geometric 
budget allocation technique. Qardaji et al. 17 survey the 
above approaches, concluding that the theoretical optimal 
fan-out of the tree is 16. They experimentally showed that 
[10], when combined with the budget allocation of [4| and 
their optimal fan-out, outperforms Privelet and SF. 

Discussion. Data-oblivious methods are fast, but may have 
low utility for practical datasets. Data-aware schemes avoid 
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this by exploiting the underlying data, but they may be pro¬ 
hibitively slow. For example, assuming the squared error 
metric, the lowest time complexity achieved by any method 
is 0(n 3 ). Attempts to boost performance via approxima¬ 
tion, by ignoring possible groupings, compromise utility in 
an unpredictable way. Moreover, the naive adaptation of 
DAWA to arbitrary queries, by feeding all the 0(n 2 ) possi¬ 
ble range-sums, is impractical. 

Finally, all the discussed methods involve common compo¬ 
nents. For instance, data-aware schemes only differ in their 
grouping technique (e.g., different error metrics in the scor¬ 
ing function), whereas data-oblivious methods only differ in 
the tree fanout and the budget allocation across the levels. 
These design decisions are orthogonal; e.g., we could use 
the tree fan-out of one method with the budget allocation 
policy of another. Going one step further, novel methods 
could combine the merits of both data-aware and -oblivious 
schemes. 

Motivated by the above, in this work we formulate a prin¬ 
cipled approach, which defines the core privacy techniques 
as primitive modules. Our framework allows (i) the careful 
study and optimization of each individual module, (ii) the 
construction of efficient and effective schemes via the seam¬ 
less combination of these modules, and (iii) the effortless 
adaptation of additional modules, such as the matrix mech¬ 
anism, in our problem setting. 

3. MODULAR FRAMEWORK 

Section |3.1| formulates the concept of module along with 
related notions. Section 13.21 describes the module instanti¬ 
ations utilized to construct range-sum schemes. Section [3.3| 
demonstrates how the existing state-of-the-art range-sum 
schemes (used as competitors in our experiments) can be 
expressed in our modular framework. 

3.1 Definitions 

There are two types of building blocks in our approach: 
the module and the connector, formulated in the next two 
definitions. 

Definition 3. A module is a mechanism that takes as 
input a sensitive histogram h £ TL and a vector of public 
parameters p, and outputs a differentially private histogram 
h £ TL. The privacy level (i.e., e) of the module depends on 
h, p, and its internal mechanics. 

Definition 4. A connector is an algorithm that takes 
as input a vector of public parameters p, and either H C TL 
(i.e., sensitive histograms) or H C. TL (i.e., differentially pri¬ 
vate histograms), and outputs another vector of parameters 
p', along with sets H' C TL and H' C TL. It must obey 
two constraints: (i) it must spend no privacy budget, and 
(ii) if it takes as input some h £ TL, all its outputs must be 
consumed by modules. 

Simply stated, modules are responsible for perturbing sen¬ 
sitive data with noise, whereas connectors connect mod¬ 
ules (and optionally also other connectors). The connec¬ 
tors essentially format the data prior to feeding them to the 
modules. The public parameters facilitate determining the 
amount of noise added by a module. The second condi¬ 
tion of the connectors is due to technical purposes in our 
proofs, which will become clear later in this section. Here¬ 
after, we denote a module by M and a connector by C. 


Finally, note that a module may be further comprised of 
other modules and connectors, in which case we refer to it 
as composite. The motivation behind distinguishing connec¬ 
tors from modules is to compartmentalize the components 
related to privacy within the scope of a module, so that we 
facilitate the understanding of its privacy level and possible 
optimization. 

Definition 5. A range-sum scheme consists of a di¬ 
rected acyclic graph (DAG) of modules and connectors, and 
a query processor. It takes as input a histogram h £ TL, 
public parameters p, and privacy budget t. The DAG of 
modules and connectors outputs a structure S (e.g., a his¬ 
togram or tree) that satisfies e-differential privacy, which is 
fed to the query processor. The latter uses the structure to 
answer arbitrary range-sum queries. 

Note that the above definition can capture even iterative 
schemes, such as MWEM [9], as follows. We decompose a 
loop into modules, and then serialize the loop by repeating 
its modules as many times as the number of loop iterations. 
We do not delve into more details, as we do not deal with 
iterative schemes in this work. 

The next theorem formulates e-differential privacy for a 
range-sum scheme. Intuitively, it states that the connectors 
do not affect privacy at all. The privacy level of the entire 
scheme depends solely on the modules and, thus, it suffices 
to analyze each module individually. 

Theorem 3. Let a range-sum scheme comprised of mod¬ 
ules Mi, M 2 ,..., M r and connectors Ci, C 2 , ■■■, Ci. Sup¬ 
pose that Mi, M 2 ,..., M r work on sensitive inputs derived 
from pairwise non-disjoint (resp. disjoint) datasets, and 
each Mi satisfies a-differential privacy. Then, the scheme 
satisfies (X^i=i £i )-differential privacy (resp. (max( =1 e;)- 
differential privacy). 

Proof. We distinguish two cases, assuming for now that 
the connectors take single inputs and produce single out¬ 
puts: (i) A connector C takes as input a differentially pri¬ 
vate histogram h £ TL from a module Mi. Since C spends 
zero privacy budget by definition, its output will retain the 
privacy level of the input, independently of the computa¬ 
tions it performs. Therefore, we can devise a module M' 
that encompasses Mi and C, and retains the ei-differential 
privacy of Mi. (ii) A connector C takes as input a sensi¬ 
tive histogram h £ TL from the scheme input. By definition, 
C can only produce a sensitive histogram h' £ TL as out¬ 
put and direct it to a module Mi. Hence, we can trivially 
merge C with Mi to create a module M[ that retains the 
ei-differential privacy of Mi. 

Replicating connectors to simulate multiple inputs and 
outputs, and executing the processes described in the above 
two cases repeatedly, from a DAG of Mi,..., M r modules 
and Ci,... ,Ci connectors we can derive an equivalent DAG 
of mechanisms M [,..., M' where M' satisfies ei-differential 
privacy. Due to Theorem pi the scheme satisfies (X^i=i e *)“ 
differential privacy (resp. (max'-! ei)-differential privacy). 
□ 

As a final remark on privacy, recall the second constraint we 
imposed on the connector. If h were the input of a connec¬ 
tor C whose output was not directed to a module, C could 
have been allowed to send h to the output of the range- 
sum scheme, violating differential privacy. The constraint 
prevents this case. 
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The benefits of modularity are threefold: (i) novel schemes 
with variable efficiency and utility can be developed based 
on a small set of simple modules and connectors, (ii) given 
the privacy level of each module and using Theorem [3] we 
can easily prove the privacy of complex schemes, and (iii) the 
modules can be optimized independently, and incorporate 
potential future improvements. 

In the following, we deconstruct existing techniques into 
modules and connectors in order to investigate their perfor¬ 
mance bottlenecks, and identify opportunities for improve¬ 
ment. The internals of composite modules and schemes are 
illustrated using figures, depicting a module with a rectan¬ 
gle, a connector with a diamond, and a query processor with 
a parallelogram. 

3.2 Atomic Modules 

The three basic modules in our framework are Smooth¬ 
ing, Hierarchy Level , and Fixed Queries. These modules 
are composite, i.e., they consist of other modules and con¬ 
nectors. However, they are used as atomi(Q blocks when 
analyzing existing and novel schemes in later sections. We 
next explain each module in turn. 

Smoothing module. This module constitutes a building 
block for the data-aware techniques. It imposes an order 
on the bins of the input histogram, groups and averages 
bins, applies noise, and outputs the perturbed histogram. 
Figure [T] depicts the internal mechanics of the Smoothing 
module in more detail. Its input consists of the initial his¬ 
togram h, and public parameters p that include a vector L, 
an error metric p (absolute or squared), and three privacy 
budgets 61,62,63. Each element of L has the form ( Qi,Vi ), 
where gi is some encoding for a group of histogram bins, and 
Vi quantifies the error in due to the subsequent addition 
of noise (the value of Vi will be elaborated shortly). 


P = (L, M, £1> 62, e 3> ) 



Figure 1: Smoothing module 

The module consists of three submodules, called Order¬ 
ing, Grouping, and Noise Addition. Ordering receives the 
histogram h and budget ei, and works as in [ 22 ]; it adds 
(Laplace) noise with scale A = 1/ei to each bin value, and 
sorts them in descending order. It then forwards the noisy 
sorted histogram h D to the Grouping submodule. 

The Grouping submodule spends budget t 2 to discover 
the groups for its input histogram, considering L and p. 
L (i) describes the permissible groups, and (ii) includes er¬ 
ror values Vi that parameterize p. A permissible group gt 
can only contain contiguous bins, and is encoded simply by 
a range of elements in h D , but is independent of the corre¬ 
sponding bin labels or values. After determining the groups, 
the submodule incorporates a group id into each bin label. 
Finally, it outputs the result, which is denoted by h 0 ,au 9 . 

^The submodules and connectors of an atomic module are never used 
outside of this particular module. 


The tasks performed by Grouping are elaborated further in 
Section [4] 

Noise Addition receives the original histogram h, budget 
63 , and the output h 0 , a „ 9 of the Grouping submodule. It 
groups the bins of h according to the (augmented with group 
ids) bin labels in h 0 , a „ 9 , and averages their values. Then, it 
adds noise to the respective average with scale l/(e 3 ■ |<?i|). 
Finally, it sets the noisy average of every group gt as the 
value of the bins in gi, and outputs the noisy smoothed 
histogram h, which satisfies 63 -differential privacjJ^] Note 
that the bin labels in h incorporate the group ids of h 0 , a „ 9 . 

Ordering is ei-, Grouping is 62 -, and Noise Addition is 
63 -differentially private, and they all operate on histograms 
derived from pairwise non-disjoint inputs. Hence, due to 
Theorem [5] Smoothing satisfies (ei + 62 + 63 )-differential 
privacy. If we set ei = 0 (e 2 = 0), the Ordering (Grouping) 
submodule acts as a connector. Specifically, Ordering just 
outputs the input histogram, whereas Grouping outputs the 
best strategy without spending privacy budget. However, 
based on Definition |4] it is not permitted to simultaneously 
set ei = 0 and 62 = 0; in that case, Ordering would for¬ 
ward a sensitive histogram h to another connector. Finally, 
if we set 63 = 0, the Noise Addition submodule adds noise 
with infinite scale to each group average. Although the re¬ 
turned h contains useless values, its bin labels incorporate 
the grouping information from the Grouping submodule. 

Hierarchy Level. This is a typical component of the data- 
oblivious schemes. Recall that these methods build an ag¬ 
gregate tree on the original histogram. Every level of the 
tree can be viewed as a separate histogram. The Hierarchy 
Level module operates on a histogram of a specific tree level. 
Figure [2] illustrates its internal parts. The module receives a 
histogram h, a vector L, privacy budget e, tree height t, and 
a tree level £. It consists of two connectors (Scale Budget 
and Scalar Product), and a submodule Noise Addition. 

p = 



Figure 2: Hierarchy Level module 

In our implementation, the Scale Budget connector al¬ 
locates the privacy budget based on the tree level, using 
the method of |4] to maximize utility. It receives as in¬ 
put the triplet (e,t,f), and outputs ae, i.e., it determines a 
parameter a that scales budget e. The Scalar Product con¬ 
nector takes as input ae and public vector L, and simply 
outputs their scalar product (ae)L. This essentially dis¬ 
tributes the budget assigned for the level (potentially) non- 
uniformly over the bins. The output (ae)L is forwarded to 
the Noise Addition submodule, which adds noise with scale 
l/((ae)L[i]) to the i th bin of the histogram, and outputs the 
resulting noisy histogram h. 

2 

Contrary to LPA, Smoothing distributes the noise non-uniformly 
over the bins of h. This can be thought of as splitting h into |G| 
disjoint histograms, each corresponding to a gi £ G and, due to av¬ 
eraging, having sensitivity l/\gi\. Due to Theorem^ injecting noise 
with scale 1/(63 |pi|) renders each histogram 63-differentially private. 
Due to Theorem | 2 | Smoothing is also 63-differentially private. 
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The L parameter is selected, so that the Hierarchy Level 
module is (ae)-differentially private. In our schemes, we 
distinguish two cases: (i) L = l n , and every bin receives 
the same noise with scale l/(ae). (ii) L [i] = 0 for some 
bins, in which case the module adds noise with infinite scale. 
Observe that in both cases, the added noise achieves (ae)- 
differential privacy. 


Fixed Queries. This module is the building block of meth¬ 
ods that target at range-sum queries known a priori. It re¬ 
ceives as input a histogram h, a privacy budget e, and a 
range-sum query workload W. It executes an off-the-shelf 
mechanism such as MWEM [9] or the matrix mechanism 
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and outputs the noisy histogram h. Figure [3] shows 
the Fixed Queries module, instantiated with the optimized 
matrix mechanism submodule of |13| , used in our implemen¬ 
tation. 


P = (W, e) 



Figure 3: Fixed Queries module 


3.3 Modularizing Existing Schemes 

In this section we show how the existing approaches can 
be constructed using modules and connectors. 

Smoothing scheme. All data-aware mechanisms [20, [ 2 J 
|22[ |l3| described in Section [2.2| are captured by the scheme 
of Figure [4] which is a simple combination of the Smooth¬ 
ing module with a Query Processor. The latter receives 
the noisy histogram output by the Smoothing module, and 
replies to range-sum queries. The queries are processed by 
adding the bins falling in the query range. 


P = (L,/i,ei,«2,e3,) 



Figure 4: Smoothing scheme 


Depending on the choice of public parameters p, we can 
have the following alternative scheme instantiations: 

• With/without ordering: We can deactivate (activate) 
the Ordering submodule by setting the value of ei to 0 
(> 0). For instance, if we set (ei, £ 2 , £ 3 ) = (e/2, 0, e/2) 
we reproduce AHP 
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. Note that £2 = 0 because the 
Grouping submodule operates directly on noisy data 
and does not need to inject extra noise (i.e., it acts as a 
connector). On the other hand, if we set (ei,£ 2 ,£ 3 ) = 
(0, e/4, 3e/4), we reproduce the smoothing scheme of 
DAWA 13 . Observe that either case results in an 
(ei + £2 + £3 = e)-differentially private Smoothing mod¬ 
ule. Since this is the only module in the scheme, the 
Smoothing scheme is also e-differentially private. 


• Exact/approximate grouping: The Grouping submod¬ 
ule can be implemented either as an exact or an ap¬ 
proximate algorithm. In the first case, public param¬ 
eter L includes all possible groups of contiguous bins. 
In the second case, L contains a proper subset, which 
reduces the running time. I 11 both cases, all Vi val¬ 
ues in L are set to l/e 3 , which is the expected error 
incurred by the Noise Addition submodule. 


Absolute/squared error metric: There are also two op¬ 
tions for the error metric p utilized by the Grouping 
submodule; absolute as in [2, 13| or squared as in 20 
22 . As shown later in the paper, this choice impacts 


both utility and performance. 


Hierarchical scheme. The scheme captures data-oblivious 
methods. As shown in Figure[5] it consists of connectors Ci 
and C 2 , t Hierarchy Level modules (where t depends on the 
input public parameters), and a Query Processor. It re¬ 
ceives as input a histogram h, privacy budget e, and public 
parameters L and /, where L = 1 and / is the fan-out of the 
tre^J Connector Ci initially receives h, e, L and /. Based 
on h and /, it creates an aggregate tree, and determines the 
tree height t. It next perceives each level of the tree as a 
histogram he for £ = !,...,£. Finally, it splits the budget 
e into t budgets e/t and forwards h^ and (l,e/t,t,£) to the 
£ th Hierarchy Level module. 

The £ th Hierarchy Level module sends a noisy histogram 
hf to C 2 . The latter assembles a noisy tree T from these 
histograms and forwards it to the Query Processor. In order 
to maximize utility, in our implentation the Query Proces¬ 
sor answers range-sum queries by combining nodes from the 
noisy tree using the method of 10] (see Section [2.2|. 


P = (L = 1, / — 16) 



Figure 5: Hierarchical scheme 

Each Hierarchy Level module £ offers a^e/t-differential 
privacy. Moreover, the modules work on non-disjoint sen¬ 
sitive inputs. As such, due to Theorem [3] the Hierarchi¬ 
cal scheme offers (X^=i ^^-differential privacy. Note that 
the £ th Hierarchy Level module sets its at as defined in [ 4 ] 
(through a closed formula based on t, £), in a way that guar¬ 
antees that X^=i a i = t- Consequently, the Hierarchical 
scheme satisfies e-differential privacy. 

DAWA-like scheme. This scheme is a generalization of 
DAWA 13 . Recall that, in addition to a smoothing stage, 
DAWA employs the matrix mechanism, which receives as 
input all the possible range-sum queries. We abstract these 
two stages, so that any smoothing and fixed-queries scheme 
can be combined to realize DAWA’s concept. 

Figure [6] depicts the mechanics of the scheme. It consists 
of modules Smoothing and Fixed Queries, a connector, and 

3 

In our implementation, we set f — 16 because it is optimal in terms 
of utility for range-sum queries [17|j. 
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p = (W,L,p,e/4,3e/4) 


sum query. However, they are in general good indicators of 
accuracy and their minimization is likely to maximize utility. 



Figure 6: DAWA-like scheme 


a Query Processor. It receives as input histogram h, budget 
e, and parameters (W, L, p, e/4, 3e/4). Following [L3], bud¬ 
get e/4 is allocated to the Smoothing module, and 3e/4 to 
Fixed Queries. Vector W holds all possible 0(n 2 ) range-sum 
queries; p defines the utilized error metric by the Smooth¬ 
ing; L contains the permissible groups. For each group gi in 
L, Vi is set to 4/(3e), which is the expected error due to the 
subsequent noise addition by the Fixed Queries module. 

The Smoothing module takes as input h and parameters 
(L, p, 0, e/4, 0), and outputs a noisy histogram h 9 that incor¬ 
porates the group labels. Connector G receives h 9 , W and 
h. It smooths h using h 9 , and creates h a „ 9 . Then, it modi¬ 
fies workload W to W a „ 9 to reflect the queries on h a „ 9 . The 
technical details of this conversion are included in [13]. Fi¬ 
nally, it feeds (h„,„ 9 . W a v g ) to Fixed Queries, which receives 
budget 3e/4. This module computes and forwards a noisy 
histogram h to the Query Processor, which answers range- 
sums by summing the bins included in the query range. 

The Smoothing module satisfies e/4-differential privacy, 
while the Fixed Queries module 3e/4-differential privacy. 
Both work on non-disjoint inputs and, therefore, the whole 
scheme satisfies e-differential privacy. 


4. GROUPING AND METRICS 


The Grouping submodule of Smoothing determines the 
way the bins are privately grouped. In all existing schemes, 
this is modeled as an optimization problem where the result¬ 
ing grouping must minimize a certain error metric. In this 
section, we first present in detail the two error metrics used 
in the literature, namely absolute [2 13 and squared 


20 


22 error, explain their usage, and analyze the overall time 


complexity of Grouping in each case. Next, we introduce an 
optimal way to compute the squared error, which (i) reduces 
the time complexity of the current best method by a factor 
of n, and (ii) improves the accuracy of Smoothing. 

Recall that Grouping takes as input a histogram h 0 , a 
privacy budget £ 2 , public vector L, and an error metric p. 
Its goal is to find the groups that minimize p, while satis¬ 
fying ^-differential privacy. Let G be a grouping strategy , 
i.e., a set of |G| groups of contiguous bins that cover all 
histogram bins and are mutually disjoint. Let bj be a bin 
value, and gi the average of the bins in group gi € G, i.e., 
S4=£ly €94 MSi|. 


The total error has two components. The first is due to the 
smoothing process and depends on the difference between 
the value bj of a bin and the average gi of the group in 
which it belongs. The second component is due to the noise 
injected by the module that succeeds grouping. For each 
group g-i, L contains a value v z that corresponds to the latter. 
The absolute and squared error metrics combine the two 
components in different ways. Both metrics represent the 
collective error per bin, rather than the final error in a range- 


Absolute error. This metric is 


defined in [2 
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as: 


|G| / 

ern ! fo J ~9i I +Vi 

i=l \bjegi 


(1) 


The state-of-the-art algorithm that uses the absolute error 
is the Smoothing module of DAWA 113], which works as fol¬ 
lows. It first calculates the cost Ci = 2Jb g 9 I bj — 9i\ -\-Vi of 
each group g, in Equation [l] by utilizing a binary search tree 
in O(logn) time. Then, it adds noise with scale l/(e 2 |< 7 i|) 
to d producing fy = J2 bj e gi I h i - Si\ + Vi + Lap(l/(e 2 \gi\)). 

Finally, it finds the groups that minimize efri = ci 

using dynamic programming in 0(n 2 ) time. The authors 
prove that an optimization algorithm that operates with 
such noisy costs ensures £ 2 -differential privacy. The total 
time of Grouping is dominated by that of computing the 
costs of all the 0(n 2 ) groups, which is 0(n 2 logn). 


Squared error. This metric is defined in [20, 22 


|G| / 

err 2 = ( 6 J “ 9if + 

i=l \bjGgi 


( 2 ) 


The state-of-the-art grouping algorithm that utilizes the 
squared error is AHP [22], which works as follows. It adds 
noise with scale l/e 2 to each bin of the initial histogram, 
and computes cost ci = J2b-eg- Ob' — <?0 2 +«?, where bj is a 
noisy bin value, and gi the average of a group of noisy bins. 
Finally, it finds the groups that minimize efr 2 = ci. 

The algorithm satisfies £ 2 -differential privacy because it op¬ 
erates on values perturbed with noise scale l/e 2 . Its time 
complexity is 0(n 3 ). 

The following theorem provides a lower bound on the time 
complexity of Grouping, in the case that all possible groups 
of contiguous bins are considered. The lower bound applies 
to both error metrics. 


Theorem 4. A grouping algorithm on a histogram with 
n bins runs in fl(n 2 ). 

Proof. The number of all the possible groups is 0(n 2 ). 
This is because we have n groups of size 1, n — 1 groups 
of size 2, and so on (recall that a permissible group can 
only consist of contiguous bins). Thus, the total number of 
groups is n + (n — 1) + (n — 2) + ... + 1 =* n{ ' n + 1 '> . it suffices 
to prove that there is an input for which any algorithm must 
check all the possible groups at least once. 

We build a histogram such that every group gi contributes 
cost Ci = \gi\ (i.e., equal to its cardinality) to the error 
metric. In this scenario, any grouping strategy G minimizes 
the error metric, since every G leads to error X ^ 9 eG c* = n - 
Now suppose that we reduce the cost of a random group Qj 
t° (Iffil — <5) for some $ > 0. Any grouping strategy that 
includes gj will result in error n — S, whereas any other 
will result in n. Therefore, the grouping strategy G* that 
minimizes the error metric must include gj. Since gj is a 
random group, the algorithm that finds G* must check the 
Ci of every group gi in order to find gj. This concludes our 
proof. □ 
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We next present an algorithm that minimizes the squared 
error err 2 in 0(n 2 ). Therefore, due to the lower bound 
in Theorem [4] our algorithm is optimal. Given that cp = 

— 1 , we observe that the cost of each group can be 
rewritten as follows. 


Ci 


E (£■ - Si) + «? = E 




\9i\ 


1 

+ Vi 


Based on the above equation, we can efficiently compute 
the cost of each group using the following procedure. Ini¬ 
tially, we add noise with scale l/e 2 to every histogram bin. 
In a pre-processing stage, we build vector vi that stores the 
noisy bin values bj, and vector V 2 that stores their squares 
b 2 . Subsequently, we construct the prefix sums for each vec¬ 
tor. Specifically, the prefix sums for vi (V 2 ) is a vector 
v'i (vi), such that v[[j] = £i=i vi[i] (v' 2 \j] = ELi v 2[*])■ 
The pre-processing takes 0(n) time. For each group gi over 
contiguous bins l, l + 1 ,..., u, we can compute Yli-eg- as 


v([w]—v([Z— 1] and^g 6gi b 2 as vi[u]—vi[f—1] in 0(1) time. 
Thus, the cost of any group requires 0(1) time. Since there 
are 0(n 2 ) possible groups, we can calculate all their costs 
in 0(n 2 ). Finally, in order to find the grouping strategy 
that minimiz es e fr 2 , we employ the dynamic programming 
which runs in 0(n 2 ) time. Therefore, our 
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procedure of 

algorithm has total running time 0(n 2 ). 

We conclude this section with an improvement on the ac¬ 
curacy yielded by the use of the squared error. Recall that 
our algorithm computes the group costs on the noisy his¬ 
togram in order to ensure e 2 -differential privacy. Thus, the 
grouping strategy that minimizes err2, may not minimize 
err 2 (defined on the original bins). In order to alleviate the 
effects of the extra noise in err-2 we exploit the following 
observation. Using a similar approach as in the proof of 
Lemma 1 in [20j, we can show that each group is expected 

to have its cost increased due to noise by 2 ^ 9i Ifi 1 , i.e., pro- 

e 2 

portionally to the group size. The additional error leads to 
smaller groups for err 2 minimization, compared to err 2 . To 
mitigate this, we reduce the calculated cost ci of each group 

gi by 2 ^ gi ^ ii ~ 1 , before feeding it to the dynamic programming 

e 2 _ 

procedure. Compared to its direct competitor AHP ;22|. 
our algorithm improves the utility by up to 70% and the 
complexity by n. 


5. NOVEL SCHEMES 

We design two schemes based on our modular framework. 
The first, called Subtree Smoothing, constitutes the first ap¬ 
proach that seamlessly combines smoothing with aggregate 
trees, running in O(n) time. The second, called Smoothed 
Prefix Sums, reduces the time complexity of DAWA by a 
factor of n, while maintaining its utility. 

5.1 Subtree Smoothing Scheme 

Recall that the Hierarchical scheme builds an aggregate 
tree in order to compose the range-sum answer from a small 
number of noisy values, thus reducing the error resulting 
from noise aggregation as opposed to LPA. However, due to 
the publication of multiple non-disjoint histograms (one per 
level), it must add more noise per level than LPA. On the 
other hand, the Smoothing scheme reduces the sensitivity of 


a set of bins via grouping and averaging, thus lowering the 
required noise. Our Subtree Smoothing scheme builds an 
aggregate tree similar to the Hierarchical scheme (thus re¬ 
ducing the error from noise aggregation), but smooths entire 
subtrees via grouping and averaging similar to the Smooth¬ 
ing scheme (thus reducing the per-level, per-bin noise). 

Figure [7] illustrates the main idea. The scheme runs the 
Smoothing module only once for the leaf level (i.e., for h), 
setting as permissible groups only those that correspond to 
the leaves of full subtrees. Suppose that the black nodes in 
the figure comprise a group in the returned grouping strat¬ 
egy. We refer to the root of the subtree corresponding to 
a group as the group root. Next, the scheme creates the 
aggregate tree, pruning the nodes under the group roots 
(black nodes). Subsequently, it feeds each level of this ag¬ 
gregate tree to a Hierarchy Level module, which outputs 
a noisy histogram. The final noisy histograms comprise a 
noisy tree. Finally, the scheme puts the pruned nodes back 
to the tree, deriving their values from their corresponding 
group root. Specifically, the value in the group root is dis¬ 
tributed evenly across the nodes of the same level in the 
subtree. This is equivalent to smoothing the nodes at each 
level of the subtree via averaging. 



Figure 7: Subtree smoothing example 


Figure [8] illustrates the modules of the Subtree Smooth¬ 
ing scheme. There is a single Smoothing module, t Hier¬ 
archy Level modules, two connectors, and a Query Proces¬ 
sor. The input of the scheme includes the sensitive his¬ 
togram h, privacy budget e, and public parameters p = 
(L, p, e/4,3e/4, /). L is the set of permissible groups for the 
Smoothing module and their associated v. t values; p is the 
error metric; e/4 is the budget allocated to the Smoothing 
module; 3e/4 is the budget distributed evenly to the t Hier¬ 
archy Level modules; / is the fan-out of the aggregate tree, 
and t is its derived height. Following 17 , we set / = 16 . 


p = (L, p, e/4,3e/4, /) 



Figure 8: Subtree Smoothing scheme 

The Smoothing module takes as input h and parameters 
(L, p, 0, e/4,0), and outputs a noisy histogram h g . L con¬ 
tains the groups formed by bins that can be leaves of full 
subtrees in the final aggregate tree. Their Vi values are all 
set to 4f/(3e). Ordering must always be deactivated because 
the final aggregate tree is built considering the order of the 
bins in h. If Ordering were activated, Grouping could select 
a group gi, whose bins are not the leaves of a full subtree on 
h (since Ordering may permute the bins of h). Therefore, 
gi could not determine a group root to smooth a subtree, 
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thus violating the scheme. The Grouping submodule takes 
budget e/4. The Noise Addition submodule receives 0 bud¬ 
get; the returned h 9 contains only the grouping information, 
which is used later. 

Connector Ci receives h, h 9 , budget 3e/4 and fan-out /. 
It builds the aggregate tree utilizing h, h 9 , /, and considers 
each tree level £ as a histogram he to be sent to the I th 
Hierarchy Level module. For every pruned node j in the 
aggregate tree (i.e., black node in Figure [7| at level l, it 
sets its scalar to L e[j] = 0, and for any other node j' it sets 

U[j'} = 1. 

The £ th Hierarchy Level module receives the histogram . 
For each bin 6, in he, it adds noise with scale — , t . , . If b 7 
is a node to be pruned, then L e[j] = 0, and bj is perturbed 
with infinite noise, while a special annotation is added to 
its label. This is essentially equivalent to completely dis¬ 
regarding the pruned nodes. Otherwise, L e[j] = 1 and the 
module adds noise with scale a At 3e ■ This procedure ensures 
that each Hierarchy Level module satisfies -differential 
privacy (by a direct application of Theorems [l] and [2j|. 

Connector C 2 receives the noisy histograms from the Hi¬ 
erarchy Level modules. First, it assembles a noisy aggregate 
tree from the histograms. Next, it substitutes the values of 
the nodes that received infinite noise in the Hierarchy Level 
modules, with the values derived from their group root, as 
we explained in the context of Figure [7] Finally, it forwards 
the resulting tree T to the Query Processor, which answers 
range-sum queries using the technique of 10'. 

We next analyze the privacy of the scheme. The Smooth¬ 
ing module spends budget e/4 and satisfies e/4-differential 
privacy. Each of the t Hierarchy Level modules satisfies 
-differential privacy as explained above. Moreover, all 
the modules work on non-disjoint inputs. Due to Theo¬ 
rem [3] and recalling that the ae values are selected accord¬ 
ing to [i] such that 1 = the whole scheme satisfies 

e-differential privacy. 

Finally, the running time of the scheme depends on the 
error metric p,. Observe that the number of groups exam¬ 
ined by Grouping is equal to the number of nodes in the 
aggregate tree, i.e., O(n). For the case when p is the abso¬ 
lute error, the running time of Grouping is 0(n log n), using 
the smoothing algorithm of 13 . If p is the squared error 
metric, the complexity is 0(n) using our optimal algorithm 
from Section [4] Each Hierarchy Level module runs in time 
linear in the number of input nodes, thus, all the t Hierarchy 
Level modules run collectively in O(n). In our experiments, 
we demonstrate that the error metric in Subtree Smoothing 
does not significantly affect the utility. Hence, we fix p to 
the more efficient square error, which yields total running 
time 0(n). 

Utility Optimization. Instead of completely disregarding 
the nodes of a pruned subtree, we can actually utilize them 
to reduce the noise of its root. Specifically, for each level of 
the pruned subtree, we sum the node values and add noise, 
producing a noisy estimation of the root. Subsequently, we 
use the average of these estimations as the root noisy value. 
The mechanism then proceeds as described above, i.e., the 
root value is distributed evenly among the subtree nodes. 
This reduces the squared error of the root value by t' , where 
t' is the height of the subtree. We omit the proof due to its 
simplicity. 


5.2 Smoothed Prefix Sums Scheme 

This scheme is based on prefix sums 11 . A prefix sum 
query over h is simply described by an index j, and re¬ 
turns the sum of bins bi ,..., bj, i.e., JV. h[i]. There are 
n prefix sums, hereafter represented by a vector s such that 
s\j] = h[i] for j = 1 Moreover, observe that 

any arbitrary range-sum query can be always computed by 
the subtraction of exactly two prefix sums; for instance, 
range-sum [ii,i u ] is answered as s[i„] — s[ii — 1]. 

The Smoothed Prefix Sum scheme takes advantage of the 
fact that there are n prefix sums, as opposed to 0(n 2 ) pos¬ 
sible range queries, to improve the complexity of DAWA- 
like methods by a factor of n. It considers the prefix sums 
as the fixed workload W, and produces a noisy histogram 
h. The latter enables the computation of a vector of noisy 
prefix sums s, such that s[«] = £V =1 h[i]. Vector s is fed 
to the Query Processor, which computes in 0(1) time any 
range-sum [ii,i u ] as s[i„] — s [ii — 1]. Since the Fixed Queries 
module leads to highly accurate s[i], the range-sum result is 
expected to have very low error. 

Figure [9] depicts the Smoothed Prefix Sums scheme. The 
only differences with respect to the DAWA-like scheme of 
Figure [6] are (i) the workload W, which now contains the 
prefix sums, and (ii) an extra connector O 2 before the Query 
Processor, which converts the output histogram h into a 
prefix sums array s. 


p = (W, L, /(, e/4,3e/4) 



Figure 9: Smoothed Prefix Sums scheme 

The Smoothing module satisfies e/4-differential privacy, 
while the Fixed Queries module 3e/4-differential privacy. 
Both work on non-disjoint inputs and, therefore, the whole 
scheme satisfies e-differential privacy. Its time complexity is 
0(|W|nlogn) = 0(n 2 logn), since now |W| = n. The ex¬ 
pected error is at most two times larger than that of DAWA 
because Smoothed Prefix Sums subtracts two noisy values 
from the prefix sums array to answer a range query, while 
DAWA essentially returns a value for the same range. How¬ 
ever, in our experiments we demonstrate that the utility of 
Smoothed Prefix Sums is practically the same as that of the 
DAWA scheme. 

A remark concerns allocation of budget t to the various 
modules. In this work, we followed the empirical allocation 
policies of the existing schemes. Determining the optimal 
allocation is out of our scope, but we consider it as an in¬ 
teresting problem for future work. Finally, except for LPA 
and the Hierarchical scheme, where the expected error is ex¬ 
pressed theoretically, the rest of the schemes are highly data- 
dependent. Therefore, their utility must be experimentally 
evaluated under different real settings, a task we undertake 
in the next section. 

6. EXPERIMENTAL EVALUATION 

In this section we evaluate the methods of Table|7]in terms 
of utility and efficiency. LPA corresponds to the Laplace Per¬ 
turbation Algorithm. H implements the Hierarchical scheme, 
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Table 1: Summary of schemes 


Scheme 

Abbrv 

Time 

Laplace perturbation algorithm 

LPA 

0(n) 

Hierarchical scheme 

H 

0(n) 

Smoothing with absolute error metric 

Si 

0(n 2 log n) 

Approximate Smoothing 

s 

0(n log 2 n) 

Smoothing with squared error metric 

S 2 

0{n 2 ) 

Smoothing with ordering 

So 

0(n 2 ) 

DAWA 

DAWA 

0(n 3 log n) 

Subtree smoothing 

SUB 

O(n) 

Smoothed prefix sums 

SPS 

0(n 2 log n) 


using all optimizations of 17 . Si incorporates the smooth¬ 
ing algorithm of DAWA 13 , based on the absolute error 
metric. S is the approximate version of Si that consid¬ 
ers only a subset of the possible groups [l3j. S 2 applies 
smoothing using the squared error metric, and it utilizes 
the quadratic algorithm and utility optimization described 
in Section [ 4 ] S 0 orders t he bin values [22], and then uses 
S2 for smoothing. DAWA 13] is implemented with input all 
the possible range queries. SUB is our Subtree Smoothing 
scheme based on the squared error metric as it offers similar 
utility to the absolute metric, and is faster by a log n factor. 
It also contains the utility optimization technique described 
at the end of Section [5.11 SPS is our Smoothed Prefix Sum 
scheme. For both SPS and DAWA, we use the absolute error 
metric, but the choice of metric does not affect either their 
utility, or performance. 

Our evaluation includes all the dominant techniques in 
their respective settings. Specifically, the smoothing mod¬ 
ule of DAWA (Si) offers better utility and time complexity 
than previous methods that are based on the absolute error 
metric and check all possible groupings jl3j. Its approximate 
counterpart S also dominates its competitor P-HP, which in 
turn has been shown to outperform EFPA and SF [2] (details 
about these methods can be found in Section [2.2[ ). Among 
the exhaustive techniques based on the squared error, the 
state-of-the-art is AHP [22], which is dominated by S 2 in 
terms of running time and utility, as explained in Section [f] 
Moreover, the approximate methods using the squared error 
metric have the same quadratic complexity as S2, while at 
best they can reach the same utility. Finally, the optimiza¬ 
tions incorporated in H have been shown to yield the best 
hierarchical method in the survey of 17 . 

We implemented all methods of Table [T] in Java, and con¬ 
ducted experiments on an Intel Core i5 CPU 2.53GHz with 
4GB RAM, running Windows 7. Following the literature, 
we assess utility using the Mean Squared Error (MSE), fix¬ 
ing e = 1. The cardinality of the range-sum queries varies 
between 10% and 50% of the input histogram size. Every 


reported result is the average of 100 executions, each con¬ 
taining 2000 random queries of the selected cardinality. 

We used three real datasets, henceforth referred to as Ci¬ 
tations [8], Rome l], and Go Walla [ 3 ]. In Citations, we 
created a histogram of 2414 bins as in [17], where each bin 
bi is the number of papers cited i times.range-sum query 
[ii, i u ] returns the papers cited between ii and i u times. The 
Rome dataset consists of 14420 bins, where each bin bi is the 
number of cars on a specific road at time instance i. A range- 
sum query asks for the traffic at this road segment during 
a time interval. Finally, Go Walla consists of user check-ins 
at 2791 locations. We sorted the locations in ascending or¬ 
der of their ^'-coordinates as in 17], and viewed them as 
histogram bins. A range-sum query returns the number of 
users in a vertical geographical strip. 

Citations, Rome, and Go Walla feature considerably dif¬ 
ferent distributions, depicted in Figures 10(a as 


and 


10(c)| respectively. Citations is very sparse, and its consec¬ 


utive bin values are similar, especially for bins that corre¬ 
spond to numerous citations (most such bins have 0 values). 
Rome exhibits high fluctuations at specific contiguous bins 
(reflecting peak hours), and includes numerous small val¬ 
ues (reflecting non-peak hours). Finally, Go Walla contains 
almost random values, since the number of check-ins is in¬ 
dependent of the value of the ^-coordinate. 

Figure 11(a) plots the MSE for Citations, when varying 
the range size (expressed as a fraction of the number of 
bins). SPS and DAWA achieve the highest accuracy. The 
error of Si, S, and SUB is up to two times higher, while that 
of H, LPA, S2 is more than an order of magnitude larger. S 0 
exhibits the worst performance because the noise injected 
by ordering yields a poor grouping strategy. The low MSE 
of SPS and DAWA is mainly due to their effective combina¬ 
tion of smoothing and the matrix mechanism. Their almost 
identical error confirms our claim in Section [5.2| that feeding 
prefix sums to the matrix mechanism of the Fixed Queries 
module (SPS) leads to the same practical utility as providing 
all the possible ranges (DAWA). In general, all smoothing 
techniques perform well because consecutive bins have sim¬ 
ilar values, leading to groups with low error (this dataset 
yields a small number of large groups). This also explains 
the marginal difference of Si and S; S can easily find a good 
grouping strategy even though it does not explore all possi¬ 
ble groups. In contrast, S2 performs worse than Si and S, 
as the squared error metric is sensitive to some small fluc¬ 
tuations in the dataset, which leads to unnecessarily small 
groups. Methods that do not rely on aggregate trees (i.e., 
LPA, Si,S, S2 and S 0 ) are affected by the range size, as the 
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Figure 11 : Citations 




Figure 12 : Rome 



number of noisy values participating in the calculation of the 
range-sum (and, thus, the resulting error accumulation) in¬ 
creases linearly with the size. On the other hand, the range 
has small effect on the utility of hierarchical methods, which 
increases logarithmically with the range size. 

Figure [1 1 (b) | evaluates the CPU-time as a function of the 
data size. In order to reduce the data size to a percentage 
x%, we select the first x% values and the corresponding bins. 
H and LPA are the fastest methods as expected by their lin¬ 
ear complexity. SUB is slightly more expensive because of 
the additional smoothing step at the leaf level of the aggre¬ 
gate tree. The next method in terms of efficiency is S, with 
complexity 0(nlog 2 n), followed by the quadratic S 2 and 
S 0 . Si and SPS have almost the same running time due to 
their identical complexity 0(n 2 logn). DAWA (0(n 3 logn)) 
is more than an order of magnitude more expensive than 
any other method. 

In order to demonstrate the utility-efficiency trade-off, 
Figure 11(c) plots the error (x-axis) versus time (y-axis), 
when fixing the range-sum size to 30% of the bins and using 
the entire dataset. The best solutions on both aspects lie 
closest to the axes origin. Although SPS and DAWA feature 
the best utility, they are also the most expensive. However, 
DAWA is dominated by SPS, which is much more efficient. 
On the other hand, fast methods such as LPA and H incur 
high error. In between the two extremes lies SUB, which is 
almost as fast as H and LPA, but exhibits 3.5 times lower 
error than H and an order of magnitude lower than LPA. 

Figure 12(a) assesses the utility of the schemes on the 
Rome dataset. The results for DAWA are omitted, since it 
failed to terminate within a reasonable time (in fact, we es¬ 
timated that it would take approximatel y three months to 
finish for this dataset). Similar to Figure 11 (a 3 SPS is the 
best scheme, reducing the error of the next best solutions (H 


and SUB) by up to 70%. H 
error. Compared to Figure 11(a) 


and SUB have almost identical 
smoothing-based tech¬ 


niques have inferior performance because, due to the high 
fluctuations of the dataset, it is difficult to find effective 
grouping strategies. As opposed to Citations, Rome yields 
a large number of small groups. Si, S2 and S achieve gains 
through smoothing only for small ranges. S2 outperforms Si 
and S because the squared error distinguishes small fluctu¬ 
ations, whereas the absolute error erroneously merges small 
groups into larger ones. Similarly, S results in up to 50% 
worse error than Si and up to an order of magnitude worse 
than S2 because the arbitrary set of the examined groups of 
the former does not include the groups that minimize the 
error. Finally, S 0 and LPA are the worst techniques. 

Figure 


The resu' 


12(b) measures the CPU-time versus the data size. 


ts and the relative order of the schemes are consis- 
DAWA can only run for up to 40% 


tent with Figure 11(b) 
of the dataset. Figure 1 12 (c) | plots the error versus efficiency 
for Rome. Again SPS and LPA lie at the two extremes of 
utility and efficiency, respectively. SUB and H provide the 
best trade-off, dominating all the schemes but SPS. 

Figure 13(a)| depicts the MSE on Go Walla as a function 
of the range size. Go Walla features almost random bin val¬ 
ues. Consequently, smoothing spends privacy budget, with¬ 
out finding groups that lead to noise reduction. The schemes 
that depend solely on smoothing, i.e., Si, S2, S 0 and S, are 
outperformed even by LPA. On the other hand, SUB, DAWA 
and SPS are more robust to the dataset characteristics, since 
they inherit the benefits of the aggregate tree and matrix 
mechanism, respectively. For this dataset, a simple aggre¬ 
gate tree generated by H is the best method. 

Figure 13(b) plots the CPU-time. The relative perfor¬ 
mance is similar to the previous diagrams. DAWA termi¬ 
nates because Go Walla is smaller than Rome (2791 versus 
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Figure 13: GoWalla 


14420 bins). Figure 13(c) shows the error-efficiency trade¬ 


off for GoWalla. H dominates all solutions in both aspects, 
whereas SUB lies close to H. 


Summary. Our experiments demonstrate that data-aware 
techniques lead to considerable error reductions for datasets 
that have similar values in consecutive bins. However, the 
gains of smoothing vanish in datasets with numerous high 
fluctuations. In these scenarios, data-oblivious methods are 
preferable because they do not waste privacy budget on 
smoothing. In between these two extremes fie schemes that 
integrate smoothing with other modules (i.e., SUB and SPS), 
and are more robust to the dataset characteristics. Specifi¬ 
cally, SPS performed identically to DAWA in terms of utility, 
while reducing the complexity by a factor of n to 0(n 2 log n). 
On the other hand, for time-critical applications (e.g., real¬ 
time traffic), where even SPS may be too slow, SUB achieves 
comparable accuracy, while having the lowest running time 
(O(n)) among all the data-aware methods. 


7. CONCLUSION 

This paper introduces a modular framework for differen¬ 
tially private histogram publication. We first express ex¬ 
isting methods in the framework, and identify opportuni¬ 
ties for optimization. We then design a new optimal al¬ 
gorithm for smoothing, which improves utility and reduces 
the running time of the current state-of-the-art. Next, we 
develop new schemes that combine heterogeneous privacy 
techniques, previously deemed unrelated. Finally, we exper¬ 
iment on three datasets with diverse characteristics. Our re¬ 
sults confirm that our modular approach enables the design 
of schemes that (i) are tailored to the data characteristics 
of the application at hand, and (ii) offer a desirable tradeoff 
between efficiency and utility. 
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